Undergraduate cognitive psychology students' evaluations of scientific arguments in a contrasting-essays assignment

This study investigated upper-level college students’ understanding of evidence use in quality scientific arguments. Responses to a required online ‘contrasting-essays’ assignment provided instructors with quick access to formative information about students’ capacity to evaluate the quality of argument and the use of evidence therein. Students inconsistently applied criteria for strong and weak evidence, focusing on superficial aspects of writing quality and ease of comprehension instead of the need for relevant, empirical, and disconfirming evidence. Argumentation has a central role in scientific thinking because it is the means by which scientists generate and communicate their empirical findings and causal explanations (e.g., Kuhn, 1993). Our prior work (Lippman et al., 2006) and the work of other researchers (e.g., Chinn, 2006; Sandoval & Millwood, 2005), suggests that novices, such as advanced high school students and undergraduates, have great difficulty learning to create scientific arguments (e.g., Chinn, 2006). They have particular difficulty with explaining how empirical evidence provides support for their main claims (Sandoval & Millwood, 2005). We constructed a “contrasting-essays” activity, based on the contrasting-cases activity designed by Schwartz and Bransford (1998), to introduce students to the type of scientific arguments we expected them to produce for course assignments and to simultaneously inform us about their proficiency with evaluating arguments Completed shortly before their first argumentation essay assignment at the beginning of the semester, the contrasting-essays activity asked students to read, compare, and analyze the quality of the argument and the use of evidence in two sample essays. Because we assumed students would need at least a basic understanding of the content material to evaluate written arguments, we had them work with sample essays about content covered the prior week in class and in their textbook readings (i.e., results from mental imagery studies and the theory of functional equivalence). We took great care to design both essays to have flaws so that at first glance it would not be obvious which was the best essay or why, and we assumed that an analysis of student responses would provide insight into their understanding of argument and their criteria for evaluating evidence.